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In evaluation Mse^ it often oceurs that several program 

partieipants Cexperiniantals) drop out of the program prior to eomplatian. 
Since noncomplatere genarally differ substantially from eompletere in many 
respects ,5 a control group whioh originally wae representative of tfie par- 
ticipant group will most likely not be represOTtative of eitiier ttie completers 
or noncompletars considered separately. 

This paper presents a roaKimmi likelihood pMoedure for partitioning 
a control group in such a situation into separate comparison aid>groups for 
assessing progran impacts on completers and noncompleters * The approach was 
used in evaluating the Mountain Plains Gareer Edudation Program, 
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1. Scope of the Problem 



The problem of the nonequivalent control group is probably one 
of the most urgent but yet one of the most troublesome dilemmas facing 
evaluators of aooial programs today* Even in the aase that participants are 
randomly assigned to the treatanent and control conditions # differential rates 
of (and causes of 3 attrition will often result in the emergence of some im- 
portant differences between the treated and untreated. 

For eK^ple^ the evaluation design for toe Momitain Plains Career 
Education Program called for eligible participant families to be randomly 
assigned to the program or a control group* As it turned out # approximately 
30% of the program participants failed to complete the program for a variety 
of reasons. Km might be expected^ the noncompleters differed substantially 
from completers in many important respeats. As a result^ while the control 
families were a good match to the participants prior to progr^ entry ^ they 
could not reason^ly be viewed as being representative of thm 70% of the ^ 
participants who completed the progrmi* ' 

it is well kno\m (Campbell and Stanley, 1966) that if a control 
group differs from an experimental group on any of the relevant factors 
affecting the outcome measures of performance^ the observed differences be- 
tween the groups on these ouitcome measures (or lack of observed differences, 
fotf that matter) may in fact be ar'tifaets of the pre-existing differences 
rather than representiiag the effect of the program. The problem of inter- 
pretation presented by this confounded setting quickly becomes devastating 
when we realise that there is often no way to be sure that a given method 
of dealing with the situation has accomplished tiie intended result. Lord 
(1967) has asserted: - 

* * * no logical or statistical procedure can be counted upon to 
make proper allowances for micontrolled pre-existing differences 
between groins* - 

Nonetheless r reality constraints force .evaluators to exp7.icitly deal with 
these recurring problOTS. lvaluators''must make certain MsumptionSr choose 
one or more approaches and exclude others. Moreover,, they often must doiSO 
within more or less rigid tijae constraints* 

Xn the castf.of the Mountain Plains program, follow-up data was 
obtained not only for program completers and the control families, but also 
for the participant fandlies who exited from the program prior to completion, 
This'^llQwed us, the evaluators to identify some major differences between 



the completers and noncorojpletars and to attempt to partition the control group 
into two comparison si^groups for assessing the effeet of the program on com- 
pleters and noncompleters separately. - * 

This paper addreasea the partioular iflstance of the noneguivalent 
oontrol group problem which we confronted in evaluating the itomtain Plains 
Career Education Program* We discuss the approach which we used and the 
rationale behind it * 

In the next section we provide some desoriptii/e and historical 
information on the Momitain Plains progrMi^ The extent of toe problrai posed 
hy the subatantial noncompletion rate is discussed in section 3/ Section 4 
describes our general approach to partitioning the control group and in 
section 5 we give our rationale for chtosing toe maKimim likelihood logit \ 
method over the classical discriminant analysis approach to toe problem* 
The results are stsimariMd in section 6* We conclude with some suggestions 
and implications for future research on toe nonequivalent control group 
situation* 

(Readers interested in the results of toe valuation of toe 
Mountain Plains profrMi are referred to Bale and Molitor^ IBuf*) 

2* The Mountain Plains Program 

In July ^ 1974 AJbt Associates f Inc. was awarded a 30--m©nth contract 
by the National Institute of Education (Contract NIE-'C^74^1047) to conduct 
a longitudinal follow-up study of participants and a control group in the 
Mountain Plains Career Education and limployment Program- The Mountain Plains 
Program is a residential^ family-centtired career education, employment and 
training program oriented to the needs of multi-problem disadvantaged families 
caught in the cycle of poverty* It offers an Integrated program of services 
to toe entire family based on its fundamental assmiption that in order to 
break the cycle of poverty it is necessary to address problems and needs 
simultaneously at toe fKnily level* ' The program generally takes from 6 to 
12 months to complete depending upon the particular type(s) of courses in 
which the family enrolls- 

The experimental design,, planned and^implemented by ^ the 
Services Division of the Mountain Plains Education and Economic Development 
ProgrM, Inc, called for randomised assignment to the participant and control 



groups* Alttough it is believed that the randomized nature of the experiment 
was not strictly adhered to/ the control f lilies were in fact fomid to be ' 
very similar to the participant families in most respects (Bale and Park, 1976) 
Control families however were earning somewhat less money mid consisted of 
approKimately 10% more f^ale headed families a^d unmarried parents*^ 

Since its conception in Pebrua^ of 1972 , the Itotmtain Plains 
program has served over 1200 disadvantaged families from the rural areas of 
Idaho ^ Mont^a^ Nebraska, North Dakota^ South DaJcota, and %oming, a sparsely 
populated region that encompasses nearly one-f if «i of ^e continental United 
States* Eighty percent of the f^ailies served by the program are white and 
the majority of the nonwhite families are Indian. The 1200 families 
served represents over 4500 individuals, 

the population for this paper consists of all 160 control families 
together with the 914 participant families who had exited the program by 
February, 1976, the cutoff date for pu^oses of toe evaluation, 

3' The Problem Posed by the Honcompleters 



Of Uie 914 participant families who had exited the program as of 
the cutoff date, 287 or 31% left prior to completion, A prelimlna^ dis- 
criminMt analysis on 23 salient characteristics indicated that Uiose families 
falling to complete the program differed significantly (at the ,05 level) 
from the completers with respect to (Ki) Education, CXa) Housing Status, 
(Kg) Ntmber of Children, (X^) toce, and {Kg) Income. Table 1 presents the 
standardized cosfficients associated with these variables together with the 
average scores for the completers and noncompleters , 

The completion rate was much higher for whites than nonwhites* The 
drop out rate was especially high for the 130 Indian families, 60% of them 
failing to complete the progrwi compared to a noncompletion rate of only 
27% for tht white f Millies, Noncompleters also tended to be families living 
with others prior to entering the program as opposed to renting or oraing 
their ovm housing. The completers were generally more educated and had 
slightly more children that the noncompleters, 

^hese results might lead one to speculate on possible reasons for 
resigning from the program. First, it is possible that the progrMi was 
geared more to the higher educated fellies who had more employable skills 



Table 1 



h CamparjgQn of the 754 Completer and 287 NQnaempl^ter 
Famlllag en Statistically Slqnifioant DlscriminatQrs 
and the ABaogiated Standardised Beta Weights 



Diserlminators BETA 

(K| ) Years of Edueation - 50 
(Head of Household) 

(Xg) Housing Status* »35 

{X3) Nuit^er of Childran ^30 

(Ki^) toee^(Head of Household) .27 

(Xg) Gross Family Employment Inoome ,20 



Average Value 
ComBleters Ngnoompletegs 



10.9 

84%^ 
1.8 
85%^ 
$3600 



10.1 

68%^ 
1,6 
67%^ 
$2800 



Housing status was coded as (1) own or rent and (0) live with others 



Raoe was coded as 



White and (0) Other 



This is the peroentage of oTOsrs. and renters 



This is the pereentage of White fMiilies 




at the time of pregram antiy. Also, the presentation may not have coroauni- 
cated well wito the Indian population. Begardless of the reasons for not 
rOTaining in the progrMi it is reason^ly clear that thm completers as a 
group were si^stantially different with respeot to oertain potentially mpor- 
tant criteria used to evaluate the program (post program OTployment inoome, 
oeoupational status # self conoept^ eto*) 

If toe control fMiilies were representative of the group of all 
participants, they would definitely not be representative of the two very 
different classes of participants , the completers and the noncompletera . 
Moreover/ in assessing the impact of the progrOTt on the completers (those 
families receiving the full "treateient") ideally the cc^pleters should be 
compared with only that si^group of cofttrol families who %^ould have completed 
the program had they entered* But of course no control family had entered 
the progr^ so that it was impossible to distinguish with certainty the 
potential completer from the potential noncompleter families # 

Bale and Park (1976) cpiestioned whether the control group was 
COTiparable to the participants* On the feasis of comparisons across 58 
charaGteristicsthay concluded that the controls wer6 comparable to the total 
participant group but not to either the aompletera or noncompleters considered 
separately^ This left us with the problem of finding two control groups-** 
one comparable to program cCTnpleters in tertis of pre^progrMi characteristics #> 
the other comparable to the resignees* 



4, TOe General Approach 

r We decided to partition the control fmnllies into ;two st^groups ^ 
the completer'^controls and the resignt^e^controls. Sine© these subgroups 
would be used in separate ccmparisons ^ere was no need to restrict Uie sub- 
groups to be disjoint or nonoverlapping* (In this respect the present appli-^ 
cation differs from the typical application of discriminant analysis . ) , 
It is the probability of completing the program that we were 
interested in estimating for each of the control families. We could than 
choose two cutoff probabilities. The control families whose estimated 
prob^ility of completing the program was greater than the lower cutoff would 
be the completer-controls. Those families below the upper cutoff would 
comprise the resignee-=contrpls * Thos%_between* the^two-^cutof f ..p 



would be ihciuded in both comparison subgroups. 

The approach we decided upon was as follows s 

1. Estimate the prob^ility of eOTpleting the program for a faBiily 
with given values on ttie discriminators K= {Xip lC^ V Xj , Hi^ , X5) * 

Use only the data for the 914 participants to accomplish ttiis. 
_ Formally I estimate tte probability function f where 

P(K)^ £ (Ki, Ka, Xg^K^, Xa) (1) 

ttie toplw is that f^ttilies with the same values on 

these 5 variables have Uie same completion probabilities * In other 
words 9 it assraes tiiat the probability of completing ttie program is 
determined (causally) by an exact function of the discriminators/ 

2. Assume that this model (estimated using only data from the 
participants) also holds for the control families* implicit 
assimiption here is that the control families are eKohangeable with 
the participants with respect to a) toe latent factors determining 
whether a fmaily will complete the progrMi and b) the relationship 
of these latent factors to the discriminators - 

3* Rank the control families on the estimated eompletion 
profcabilities frOTi high to low, 

4, Choose the two cutoff points discussed above so as to maKimiEa 
the goodness of match between the eompleter'-participants and com- 
pleter-controls and between the resignee-participants and resignee^ 
controls by mindUni^ing the nimtoer of significant differences on pre^ 
program characteristics • In the ease that some significant differ^ 
ences are inevit^le^hoose the cutoff points that yield significant 
differences for those same characteristics Uiat ware found to yield 
significant differences between all participants md all controls- 
Only those cutoff points utilizing at least half of the 160 control 
fMiilias would be considered in order to assure that the cofflparison 
group would be large enough for the eval^ of the progrMis to be 

considered reasonably reliable. 



To impl&mmnt £his five^atep approach we needed t©; = 

• choose the funetlonal form of the probability f 
and © choose the method of estimation. 

5* Bgtijttation of -Qie Coapletioa Probability 

- Claseioal aiicrimin«t analysis (Anderson # 1958) is based on the 
asaimption that the joint distriiutiQn of the diitfriminatQrs is multivariate 
_ nomal with the sMie eovarianoe matrisc within tiie two groups Ccompletera and 
resignees) * if these conditions ware met the oompletion prpbability would 
satisfy the multivariate logistle distributlQn (Truett ^ Cornfield ^d 
Kaimel, 1967) - ' ^ 

PCX) ^ l| '{1 + expC^gCK))} (2) 

where g(X) = a+PiKi +62^2 +B3K3 +B^X^ +65X5 (3) 

is the linear disarimlnant function, . 

The probabilities could thus be eetimatad by si^stitutlng the 
estimated discrimin^t function in place of gCX) In equation 2* These 
estijnates wuld be unoonditional maximum likelihood estimates imder .the 
nomality assusiptions* 

Mternatively, the probabilities oould be estimated diraetly by 
the conditional maKlnmm likelihood approach employed by logit analysis in 
whioh case the discr^inant function estJjiate^ if desired ^ can be obtained 
by oomputing the log odds, l,e* g(K) ^ In P(K) j (l-^P (K) ) . Hal^arln, 
Blackwelder and Verter (1970) show that extras biases c«i result such as 
estimating a prob^ility of ,9 when the true value is only ,2 if the clas- 
sical discrMinant fimotion estimation procedttfe is employed when one or 
n^re of the discrimlnatora is nonnormal but dichotomous and equation (2) 
holds* Since equation (2) holds under a family of exponential distributions, 
thay reconmiend tiia logit estimation procedure for cases involving quali- 
tative/categorical varieties* 

However, there is no reason to suppose that g(x) will be linear. 
In general, interaction effects will be present and g(x) will not be linear ■ 
in X* Without Information regarding the joint distribution of the discrimi- 
nators it was not clear how to formulate a functional form which included : 



interaction teras* 

In tiia context= of all eategsrical variables> Gootoan (1972) 
employa the nonparainatrie eoncept of a. saturated model which includes all 
poee^le interaction terms in the form of a generalised lqgistl0 ftmction* 
Maintaining equation (2)^ if all of the X*s were dichotOTOUi taking on- the 
values 1 and -1, Gootoan's representation could be expressed as follows 
(Magidson/ 1976) i 

g (X) = constat + five main effects + 10 bivariate interaction effects 
t 10 trivariate interaction effects 
+ 5 '4tii-order interaction effeeta 

, V--,:. ...... - 

+1 5 way interaetion effect 

Regardless of the true diatribution of the X *s, the estijEiated probabilities 
using the sati^ated modeJCS) , (4) and dichotomiMd K's will always equal the 
observed proportions. Various unsaturated models formed by omitting some 
of the interaction effects can be tested using a qhi square test with degrees 
of freedom equal to the niHE^er of effects omitted. While the general approacH 
can be used with any quantal response models the logistic function equation 
(2) has computational advantages and can be derived using the clasrsical 
ANOVA formulation for the effects as applied to the natural logs; of the 
cell counts (Cioodman, 1972 1 Bishop, Fienberg and Holland, 1975)." 

We decided to use e^ations (2), (4) as our model in order to 
gain the use of interaction terms* We dichotomized (Ki ) Education, (X3) 
NirndDer of Chiltoen and (X5) Income at the medians in an attempt to minimise 
the number of empty cells in the five way contingency table* The resulting 
discriminators were (E) Education (High/ Low) , (R) Race (White, Ottier) , 
(H) Housing Status (Own/rent, nOther) , (C) Nu^er of Children (2 or more, 
less than 2) and (I) Income (High, Low) , 

For ease in interpretation we included only main effects and first 
order interaction effects in our initial model,. This also served to reduce 
the scope of our model selection problem, A chi square goodness of fit 
statistic was available for testing whether we needed to hypothesise higher 
order interactions * Estimates were calculated using Goodman ' s ECTA 
(Everyone ' s Contingency Table Analyser) computer program which maximises the 
likelihood function of the multinomial distribution. 



. ... . . , . . \ . . ■ . , ■ 

Regarding the possible loss of infoOTation due to dichoto- 
mization/ after finding an initial model using the diehotomies, one oould 
always test for loss of information by e^anding the diahotomies to tri- 
Qhotomies and using title chi square statistio to test if suoh e^ansion 
eignifioantly improvea discriminatibn. The variables can eontinue to 
be expanded iM ttis manner so long as the nOTber and pattern of rapty cells 
does not oause the model to be under identified. We felt that diohotomiea 
would probably be suffioient for our pu^osea and the results could be 
simmarized by the thirty- two classes of profiles formed from the five way 
claasifioation sohems* Biis would provide us with a oonvenient method 
for presenting the results in tabular form whioh could be eaaily scanned to 
determine whether the results appeared reasonable. 

6* Partitioning the Control Group 

In Table 2, the 32 profiles of participant fMiilles are ranked from 
high to low on the estunated prob^ility of COTipleting the Mountain Pla:^s 
Program* The higher educated White families who owned or r^ted their 
housing were estimated as being most likely to complete the progrm, A 
Gompariaon of the estimated probabilitiea with the actual aompletion ra tea 
obseru^ed show a reasonably good match* Notiae that the few l^ge disarepanoies 
which do appear occur for prof lies cont^^ fewer than three participant 
.familiea, ' ' \ ^ : ^r^' - ^'^^ ^■^•■^ r.^ .-...^ ^ . - . /.^ ; 

Table 3 displays aummary statistics for the 9 models est^ated/ 
These nUDdels were selected by thm procesa of backwards elimination as 
diaoussed by ^o^aaan (1971) * For our purposes ^ model H^V the model with 
the highest probability value was choaahT For a given degree of parsMony 
MB meaaured by toe degrees of freedom^ the higher the prob^ility value the 
closer the fit to XAim observed proportions* 

A comparison of model with the main-effects-only models model 
Hg^indicatea that the interaction effects included in model H^aignificantly 
improve the fit to -Uie obsen^ed proportiona. ^e difference between the 
chi-se^are statistics associated with these models has a chi-square diatribu- 
tion with 5 degrees Of freedom (the difference between the degreea of freedc»i 
associated with the two models) under the null hypothesis that the main- 
ef f eets-only Mdel is correct. The difference between -the chi^square values 



Table 2. 



PfQbablllty of Participants Completing the Program for Each of 31 Profiles of 
Charaeteriitlcs Based on Education and Race of Family Head* Housing 
Status, -Number of Children and Gross Family Employment Income 







PROFILE OF CHARACTERISTICS 














EDUCATION 
n .^h1gh) 


(R) 
RACE 
(1^ white 1 


(H) 

HOUSING 
(l^own/rent) 


(c) 

# CHILDREN 
1 = i 2 


(I) 

INCOME ' 
(1 > high) 




PROBABILITY 
OF 

COMPLETING 






1^ ^-low) 


(0 other j 


(0 * other) 


0 - < '2' 


(0 - low) 


n 


iSTlMATED 


OBSERVED 








1. 








1 


1 - _ 
0 


821. 






/ 




2. 


1 


1 


1 


1 


54 


*86 








3. 


1 


1 




0 


1 


83 


,86 


*87 ^ 






4. 


1 


0 




1 


0 


10 


.as 


' -90 / 






5. 


1 


0 


1 


1 


1 


9 


,82 ~ 


,78 








6. 


1 


1 




0 .- 


' 0 


69 




. , ts/ , 








7. 


0 


1 




0 


1 


74 


.78 


,78 








8. 
9. 


1 
0 


1 

1 


0 
1 


1 . 

■.■■1 ■■■ 


1 
1 


9 

109 


,74 
.73 


.89 
.70 








10. 


1 


1 


0 


0 


1 




.69 


.82 




. tiJ 


11. 


0 


1 


0 


0 


1 


27 


.69 


.62 






12. 


1 


. 1 


0 


1 


0 


6 


.68 


,67 - 








13. 


1 


0 


0 


1 


0 


7 


.67 ^ : 


.57 








U. 


1 


Q 


1 


0 


Q 


n 




.64 






15. 


0 


1 


0 ^ 


r ^ ^ 


^ 1 




.63 


- 1.00 


Ly 

^ • " 




16. 


.. 1 




: . ^, 0 




0 


24 


.62 


,58 ' 








1 


0 


' 0 


1 


1 


1 


.62 


,00 




^ - 




18. 


0 


0 




1 


1 


8 


.61 


.75 








19. 


- Q ■ 


V 1 




0 


0 


55 


.61 


.58 








On 


1 


0 




0 


1 


.13 


.59 


.31 






21. 


0 


1 




1 


0 


72 


. . .55 


.16 






22. 


0 


0 




1 


0 


24 


.53 


' .54 






^ 23. 


0 ; 


0 


0 


1 . 


1 


7 


.50 


,43 






24. 


0 


1 


0 


0 ■ 


0 


M 


,49 


,44 






25. 


^ ^ 0 


.0 


1 


0 


1 


16 


.45 


,62 ^ 






_26. 


r ' 0 


1 


0 


1 


0 


15 


.43 


.47 






27, 


. 0 


^0 - 


0 


- 1 - 


0 


11 - 


.42 


.45 








1 


0 


0 


0 


0 


17 


.39 


,29 






2i. 


0 


Q 


1 


0 




25 ^ 




.36 






30. 


0 


.0 


0 : 


0 


1 


3 


.35 


,33 . 




- ! 


31, 


1 


0 


0 


0 


1 


2 


- '-34 


1.00 






32.- 


0 


0 


0 


0 


0 


23 




.35 


y 





Table 3 



The Main Effects and Interaction Effects Models and Associated 
Goodness of Fit Chi Square Values for Models Fit Using Data on. 

914 Known Gompl eters and Reslgnees ' ~ — 



Model 


Explanatory Variables 


Degrees 
■ nf 
Freedom 


Chi 


Square 


Probabllltv 
Value 


"o 

KJ 




n 


n 


n 

u 




M 

"l 


T H r P P PR TP TR HP 

CE,IH 


20 


20.5 


22.4 


.32 




I,H,C,E,R,CR,IE,IR.HEv 
CE 


21 


21.2 


23.3 


.33 


"3 


I,H,C,E.R,CRJE,IR,HE 


22 


23.5 


25.5 


" .27 


"4 


I,H,C,E,R,CR,IE,HE,CE 


22 


23.3 


25.0 


.30 


«5 . 


I,H,C,E,R,CR,IE,HE 


23 


25.8 


27.4 


.24 


"6 


I,H,C,E.R,CR,HE 


24 


28.7 


29.6 


.20 




I*H,C,E.R,CR 


25 


30.6 


31.6 


•17 




I,H,C,E,R 


26 


37.9 


38.3 


.06 



ie 15 (or 17 if the likelihood ratio chi-equare - is used instead of the 
Peaeon chi-iquare X^) which als sufficiently large so as to reject the main- 
effaeta^only modal in favor of the intefaotion effeots models H^- (It - 
: was also found that the model would result in 

substantially different oontrol; groups than those defined in Table 2.. The 
definition of the control groups is diacussad later in tiiis aeetion*) 

T^le 4 presents the estimated effeots assooiatad with the vari^le 
The most mportant variables for predioting program completion are IdueationV 
Baoe and Housing Statue* The relatively large negative interaction effect 
associated with Children and Race indieatas that the effect of the number of 
ohildren_upcn completing tiie program depends upon race. Specifically ^ 
since the magnitude of this InteraGtion effect e^oals the magnitude of the 
main effect associated with the nia^er of children* the model states that 
the nmODer^ of children does not influence tte decision of white families 
to complete or resign from the program, but it has a strong effect upon 
other families* 

In order to see if the interaction effect was reflected in the 
data we cross classified completers and nonc^pleters by Race mnd Kianber 
of Children* Sure enough, there was a strong interaction effect. White 
families conpleted the program at toe same rate (73%) regardless of whether 
they had few (0 or 1) or many (more than 2) children- Non^hite fmilies 
however* were much less likely to resign from the program if they had many 
Ghildrenr While 58% of non-white families with 0 or 1 child left the program 
prior to completion, only 39% of non^white families with 2 or more children 
did so* These results are displayed in T^le 5. Similar checks also 
supported the reasonability of the selected model* 

The partition of Table 2 was then applied to the control population* 
All control families having an estimated probability of collet ing at *61 
or higher comprise the cOTvpleter^control group* This is designated in Table 2 
by an upward arrow beginning at the estimated probability of *61* These _ 
control families consist of the 19 profiles most likely to complete ttie _ 
progr^ as estimated by the maximum likelihood procedure described earlier. 
An alternative cutoff prob^ilitiy of , 62 was also considered but thm resultin 
match was somewhat better for the ,61 cutoff, ^ " ' ~ 



Table . 4 



erJc 



The Main Effects and Interaction Effects Estiniated by Modal 
Hg, as Fit Using 914 Known Completers and Reslg^^ 



y Main 
Variable Effects* 



Education (E) +,202 

Race (R) - + .187 

Housing (H) +.186 

Chndren (C) +,112 

Income (I) + ,075 



Interaction 

Variable Effects 

Children/Race (CR) -.113 

Housing/Education (HE) + ,070 

Income/Education (IE) - .065 

Incoma/Race (IR) +.062 

Children/Education (CE) + .062 



* _ - - . • • 

Positive values indicate propensity to complete program. 
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Completers and NoncomplBter PartleipMts Claaaified 
by Race and Number of Children 



0 or 1 child 2 or more 



Nonwhltss . 
0 ar 1 child 



377 (100%) 350 (100%) 



110 (1Q0%> 



2 or mora 



Con^leters 


r ■ ... - 




Completars 




47 «61%) 


277 (73ti) 


2B7 (73%) 




46 (42%) 


Rasigneee 


100 (27%) 


93 (27%) 


Reei^ees 


64 rm) 


30 (39%) 



77 (100%) 



17 



A similar proeedt^s was used to define the risignae-eontrols . 
This Lm design^tirf by the downward pointing arrow below .85. Other 
prob^ility^eutof^s considarea for the resi^ee'-Gontrola which led to gome 
woree matehes ware -80^ .73 and •70# 

^ . ._ goal to he accomplished by " the partitioning of the control 

group was - to arrive at comparison groups which are comparable (exchange^l#). 
with the associated jparM.cipant groups (completers and noncompleters) with 
respect to causal frictorfj determining the outcome meaaurea upon which the 
progrmn would be evaiuated. Altoough these factors are unknovm and/or un-* 
measured one might make inferences ^out these compariaons from the gootoeaa 
of match between the groins on pre-progr an characteristics. The better the 
match on pre^p^gran oharacteristios the more likely that tile comparison 
groiips are similgir on related unmeasured variablea which effect the various 
outcome measures* . 

■ Table 6^1 through 6--^4 diaplay the pre'-progrMi comparisons for all 

participants, as well as separate comparisons for conpleters and reaigneea 
with tlieir correaponding control groups* ^he prob^ility valuea aasociated 
with th€ con^leters and wito the resignees are TOStly substantial in a iM 
indicating quite good matches given by tiie control subgroupa as defined in " * 
this section, '^^he match for the completers is somewhat better than toat . 
for the reaigne^ae* By selecting appropriate si^sets of the control fMiliea v 
by the methods deacribed earlier we have obtained what appears to be reason- 
ably good cc^parison groups for the completers and reaigneea, ^ 

Let us now gamine a few of these co^arisona in detail. Notice 
. firat that there are good matches (nonsignificant differences) between tile 
completer-controls and reaignee-participants with respect to the five variabiles 
used in defining these control aubgroups (i.e.. Education ^ and Hace of Head , 
Houaing Status, Number of Children and Total Family Inc^e) * For each of 
these five variables the use of the contror subgroups is a si^stantial 
improvement over the use of all control families as a comparison group for 
both completers and resignees* 

For eKMiple, consider -'Race." While 85% of the completers are 
white Md 67% of the resignees are white, the total control population con-- 
sists of 81% white families. By restricting the set of control families to 
the completer^control subset Bxid the resignee-control sxAset of families, 
this latter percentage becomes 91% and 74% white families respectively, which 
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gives a goad mateh ta the con^leter and reaignee participants. These results 
ara liml^ for the other four vari^les as well* 

Consider neKt "Nimter of weeks un^ployed last year , " one of the 
pra^progrMi oharaoteristicg not used in defining to eontrol subgroup, Thm 
partiaipant-oQmpletars ware unefl^loyed an average of 18 weeks while tiia 
avarage .rasipiee spant 24 weeks witiiQut a job. The averags control family 
again falls somewhere in between, at 22 weeks, Thm eompleter^control and 
resignee^control si^groups yiald figi^es of 19 weeks and 24 waaks respaetivaly, 
again an aKcellent TOtoh to the eemplatar and rasignee partipants, Thm 
rasi^ting goo^ess of matoh is similar in als^st all otiier instances where 
the overall control grot^ was fairly smilar to the group of all participant 
familias. 

Finally, let us consider a variable such as "Sax, Head of F^iily," 
on which the controls differed significantly from the participants, while 
the controls consisted of 29% female headed f andlias , females constituted 
the head of only 20% of tha participant families- This 9% difference is 
'significant at well beyond the ,05 leyal, Thm quality of match given by the 
control siabgroups is also a function of the quali'^ of match betwaan all 
participants and all controls as now will be seen* 

^a braakdown of the 20% female-headed p^ticipant fellies becomas 
21% and 17% respactively for the completer-participants md rasignee^ 
participants , Although the control families have a larger proportion of 
female headed families than either of these participant groups, the control 
subgroups still perform in the desirable'' direction. Just as thm completer^ 
particip^ts have a higher percentage of fCTiale headed fMiilies than the 
resignee-'partiGipants , the completer^control si:^group also has a larger 
percentage of female-headed families^ than ^es the resignaa^control subgroup 
{30% compared to 25%)- Thus, the significant difference between the complater- 
controls and completer-^participants on this characteristic is in part attri^ 
butable to the fact that control families consist of a higher proportion of 
female^headed famlies. 



7. Stjmnary and IgipliGatiQns for Futiare Applieatiene 
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In this paper we applied toOitaM' 8 general nonpafametric approaeh 

for the analysis of ^lalitativa/aategoriaal vari^les to classify the Gontr©! 

popoulation into estimated acmpletere and estimated resigneei from the Moun^ 

tain Plains Career Education Program* method performed well ae judged 

by similar comparisons between the eospleters and the estimated completer* 

controls and between the resigneee and tiie estimated resignee-controls as 

cos^ared to all^toei.p«^iclpants and all controls comparison, 

Recently f a nmiber of articles have compared the discrimwiant 

-i = . . , - _ ■ . 

analysis estimater witti tiiat of logit analysis (Halperin* et, al* 1970^ 

HcFadden, 1976 and Efron, 1975)* 

Although it was/ shora by Halper in f et* al. toat the discriminant 
analysis estimator can lead to severe biases when one of the discriminators 
is a dichotpmy/ all of these papers assimie no interaction effects* In tiie 
case that interaction effecte do exists both the classical discrijaAnant . 
analysis estimator and the IJjftear logit model are subject to major distor-- 
tions depending upon the OTOtmt of interaction that exists. Generally, it 
might be ej^ected that interaction effects are present and as a general 
rule one shotild test for them before postulating Uieir nonexistence , In 
this paper we uncovered the existence of a rather strong. race/n™^er of 
children Interaction effect* 

One lljaitation of the research report hare is that we did not 
compare aitemative methods for partitioning tiie control group. Another 
was that we did not examine the effect of dichotomizing toe variables. 
These omissions were due to t toe contraints- Finally # although our model 
was interpreted as deternujiing the probability in a causal sense # we did 
not adequately deal with the concept of measurement error. 

A more intuitively pleasing f emulation ndght posit the probability 
of completing the program as a causal function of (F^) Socio* Economic 
Status j(F^) Receptivity to traditional teaching methods and (F^) Propen- 
sity to remain in one place. All 5 of the observed discriminators can be 
assOTied to be indicators of F^. Race and Education can be assumed to be 
/fallible measiures of F^ while Fj might be measured by Housing Status and 
Number of Children. In Magidson (1977a)> tiie formulation of models using . ; 
unobserved constructs is recommended in dealing iwlth the general nonequivalent 
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GoatTOl grotap problem. For an application of Uiis appreach see Itegidson 
CiiTTb) . WiUi rsspact to latent steueture models r see Lazarsfald Henry 
(1968) and Goodmm (1974) . - 

We suggest tte need for research on nonlinear classification 
teolmi^es , ^ 





'Jknalyies were Gondueted eeparataly by sex and marital itatias , Kiua ^ 
married fCTale partisipantg were aampared to marrlM feg^le gentrola and 
maiaM^ried female participants ^re Qrapared to female qontrole who were 
also mmiarr led* See Bale and Molitor (1976) . 
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